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ABSTRACT 

The ALL WIN-D.S.A. Project, administered by the 
Office of the Los Angeles County Superintendent of Schools (OLACSS) , 
was to provide basic-skills instruction in reading and mathematics 
that would result in significant improvement in. the respective 
achievement domains, in learner attitudes, and cost effectiveness 
relative to the previous curriculum and instruction. The OLACSS 
selected the New Century Education Corporation to provide the major 
elements of the educational program which has three major components: 
(1) individualized learning centers; (2) a mathematics and reading 
curriculum covering all grade levels, delivered in an individual and 
prescriptive manner; and (3) a preservice training institute for 
teachers and administrators. The New Century program was installed at 
48 schools that met OEO eligibility requirements. Fiscal data for 
this study were collected from the various school districts in order 
to determine : those program costs that had been paid from district 
funds. Outcome data collected as a part of the initial ALL-WIN 
evaluation were utilized for considering the effectiveness of the 
program. Data from various of the outcome determinations are 
summarized in this report but unique analyses have been performed 
only for three of the cognitive achievement measures. (MV) 
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CHAPTER I 



INTRODUCTION 



In accordance with his contract with the. Off ice of 
the Los Angeles County Superintendent of Schools, 
Dr, Marvin C, Alkin (Educational Evaluation Associates) is 
pleased to submit this final report on the cost-effectiveness 
evaluation of the ALL WIN-U.S.A. Project during the 1974-75 
school year, 

ACKNOWLEDGEMENTS 

We are greatly indebted to many of the professional 
staffs at the district and county levels for their cooper- 
ation and for the assistance they provided. Special thanks 
are extended to Dr. Harry Holmberg for his guidance and help 
in initiating the project. Unfortunately, his call to 
military service during a portion of this study made it more 
difficult for us to call upon his wise counsel. In his 
absence, we were ably assisted by Dr. Mary Martin and by 
Mrs. Julia Payne. Mrs. Sara Bates provided assistance in 
contractual matters as well as in a fuller understanding of 
the fiscal data of the ALL WIN-U.S.A. Project as adrainistered 
by L.A. County Schools. Miss Barbara Waldrup was most 
helpful in reviewing v?ith us various fiscal data available 
at the county schools. Our thanks also go to Dr. Ruth Cohen 
for her assistance and helpful feedback on the earlier 
evaluation study which provided some of the outcome data 
utilized within this study. Finally, our appreciation goes 
to Dr. Maylon Drake for assisting in the conceptualization 



of the study and providing general encouragement. 

Each of the school districts participating in the 
ALL WIN -U.S. A, Project responded to our request for 
assistance and information in an able and timely fashion, and 
v?e are grateful. We were in the district and met personally 
with a number of the project coordinators to obtain their 
suggestions on the types of fiscal data that might be 
available and appropriate. In particular, Dean Gahre 
(Fontana Unified School District), Hector LeRoy (Colton 
Joint Unified School District), and Jack Hasinger (Ontario- 
Montclair Elementary School District) provided assistance 
and insights into the formulation of the questionnaire • In 
addition, we are grateful to ^^rs. Estella Schult?. (Compton 
Unified School District), Mr. Robert Burns (Duarte Unified 
School District), Roger Temple (Garvey School District, and 
Chester Jensen (L. A. County Special Schools) for thoir 
assistance in completing the questionnaire and providing 
other needed information. The business offices of each of 
the above mentioned school districts v?ere also most 
cooperative j.n providing data. 

BACKGROUND 

The ALL WIN-U.S.A. Project wa:&- funded by the United 
States Office of Economic Opportunity (OEO) , and admi.nis tered 
by the Office of the Los Angeles County Superintendent of 
Schools (OLACSS) • The mi&Siton of the Project v;as to provide 
basic-skills instruction in receding and mathematics that 
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would result in significant improvement in the respective 
achievement domains, in learner attitudes, and cost 
effectiveness relative to the previous curriculum and 
instruction. EEA v^as selected bj^ the OIACSS to conduct the 
independent evaluation of the ALL WIN-U.S,A. Project during 
its first year of operation, the 1974-75 school year. The 
Evaluation Report was delivered in August, 1975* Yhe 
cost^ef f ectiveness studj^ V7as funded in July, 1975. The 
timing of the funding for the cost-effectiveness study posed 
a number of difficulties ^.^hich will be discussed in a later 
section of this chapter. 

The OLACSS selected the New Century Education 
Corporation to provide some of the major elements of the 
educational program. The New Century program had previously 
been developed and installed at various locations in the 
United States. The program has three major components, as 
described below: 

1. Individualized Learning Centers - classrooms 

converted to laboratory- type classrooms 
somev;hat similar to the now ^common and 
familiar "language labs". The learning 
centers feature individual student carrels 
contai^ning audio-playback machines. 

2. Instructional Materials - a reading and mathe- 

matics curriculum covering all the grade 
levels, similar in content to typical text- 
book content, but delivered in an individual 



and prescriptive manner. 
3, Professional Training - a preservice training 
institute for teachers and administrators 
to provide them with a thorough understanding 
of the program *s objectives and methods and 
to instruct them in the operational proce- 
dures necessary to implement the program 
effectively. 

The OLACSS administered OEO contract provided the 
cost of purchase and maintenance of all Ue\^ Century equip- 
ment and materials and the costs of pre-and in-service 
education provided by the Company. Costs and expenses of 
L. A. County Staff charged with the responsibility of 
administering the program were also included. School 
Districts provided classroom space, teachers, aides, school 
district administrative support and related expenses. 

The New Century program was installed at forty-eight 
schools that mat OEO eligibility requirements. The most 
noteworthy of the school eligibility requirements are: 

A. The school must maintain an average monthly use 
of learning centers of at least 85 percent of 
capacity and 70 percent of the centers' daily 
service capacity must be students v/ho meet OEO 
low-income quidelines, and 

B. The schools mist categorize students ranked in 
the fourth quartile on a standardized achievement 
composite test as ineligible for participation. 
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C. Teachers selected must be fully credential^-d, 
with a miiiiinum of 2 years' experience in the 
district and have met or exceeded distxict 
determined performance appraisal criteria. 

NATURE OF COST-EFFECTIVENESS STUDY 

In this age of concern for accountability it becomes 
seductively appealing to raise questions related to the 
relative "cost-effectiveness" of programs. The simple 
questions, "Is this program cost-effective?" or "Are the 
educational outcomes of this program in line with the associ- 
ated costs?" are not ones that are easily answered. In fact, 
there have been very few studies performed which relate costs 
and outputs in education or in any other field » ^ 

A simplified cost^outcom.e study requires either 
maximizing outcomes or minimizing costs. That is, once v/e 
ascertain the costs and the outcomes of various program 
alternatives we either select the alternative that yields 
the largest amount of outcomes for a given cost, or we can 
select the altemativp which will yield ttie least cost for a 
given level of outcom.es. Unfortunately, it is not always 
possible to either maintain constant costs or constant out- 
comes when comparing programs. It's only infrequently that 
two programs cost precisely the same amount to operate and 
it's only under certain conditions that outcomes can be 
expressed as either having been attained or not attained^and 
thus comparable o 9 
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The typical cost-benefit study solves this dilemma 
and simplifies the problem somex^hat by converting outcomes 
to dollar equivalencies. That is, educational outcomes 
(student achievement) must be converted into an actual or 
estimated dollar ^>7orth. A cost-benefit study utilizes a 
decision rule (one less dollar of cost and one more dollar 
of benefit are of equal value) that enables the consideration 
of both costs and benefits. Therefore, cost-benefit 
decisions are made bet\7een alternatives xv^ithout the necessity 
of holding either costs or benefits constant across the 
programs . 

But public school practitioners are really not 
concerned (especially at the local district or county level) 
with performing cost-benefit analyses. One reason for this 
is that the nao^re of data at local district levels does not 
lend itself to the performance of cost-benefit analysis. 
Secondly, at local levels, we are usually unable and 
unwilling to make the conversions of educational outcomes 
into dollar benefits. \\Tiile the economic gain in earning 
power (and other such factors) related to particular 
increases in educational outcome can be calculated on five- 
year national data, those kinds of estimates are more 
difficult, if not impossible at a local school district level. 
(So tell me, Mr. Superintendent, how many dollars is it worth 
to each child in private benefits of increased earning power 
for each one month gain in reading score? And, tell me, how 
much is it worth in dollars of public b enefit to have a 
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next door neighbor who reads at one month's higher ability?) 
Cost-benefit analyses are usually left to the economist 
dealing viith highly aggregated national level, data. 

Cost-effectiveness analysis is another kind of 
attempt to utilize fiscal data in decision making. Generally, 
the procedures compare programs based upon their costs as well 
as their outcomeG without converting outcome measures into 
dollar benefits. It represents no easy task and most studies 
err in the direction of presenting analyse-, . v^ell beyond the 
capabilities of their data. Cost-ef f ecti^^eness analysis is 
primarily a methodology to assist decision makers in showing 
relationships between costs associated with an educational 
program, and outcomes, or effectiveness. The simples c 
way of considering cost is to document in tabular form the 
procedures used for determining the incremental cost of a 
specified program and documenting fully the outcomes associ- 
ated with that program without attempting to draw a relation- 
ship between the two sets of numbers but assuming that a more 
reasonable educational decision can be made based upon a full 
understanding of costs in addition to outcomes. Many 
cost-effectiveness studies should be content with such a 
simple presentation, for their data does not allow for more 
sophisticated analyses or presentations. 

A second kind of cost-effectiveness analysis that has 
been used is the simple development of a ratio heU^ccn cost 
figures and outcome data. The cost per month gain is a 
typical means of expressing this relationship. This provides 
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additional information for decision makers; however, a 
determination mast still be made as to whether the dollar 
ratio of achievement gain to cost is considered satisfactory 
(is $30 per month gain in reading ''good" or "bad"?). One way 
of handling this dilemma is, of course, by comparing t\v70 
different programs which have been pre-selected at the outset 
of the evaluation study, -and in which student populations and 
community characteristics are essentially similar. Costs 
are determined by the same method in each program and outcomes 
are measured in precisely the same manner. In this way, a 
comparison of the cost per month gain ratios of each of the 
t^^o programs provides some insight into which of the programs 
is likely to be most cost-effective. 

TS^o even more sophisticated approaches to cost- 
effectiveness analysis have been mentioned in the literature 
but to our loiowledge have not been tried other than in very 
limited instances. Tlie first of these more sophisticated 
approaches entails the prediction of student outcome scores 
based upon their pre-test scores and then uses the differences 
betX'jeen their actual attained outcome scoi-e and the predicted 
outcome score (the. regression residual) as the basis for the 
outcome portion of the cost-effectiveness ratioc This kind 
of approach has reportedly been used in a Dallas Public 
Schools' cost--effectiveness study. An even more sophisticated 
approach (described by Alkin in 1967) utilizes multiple regres- 
sion techniques with cost as one of tlie predictor variables. 
This would seem to be specially, infeasible except in 
situations of a highly controlled research context titiiizing 



nailtiple programs and selections of students determined by 
the research team* In short, not appropriate for most school 
district evaluations. 

CONSTRAINTS OF THIS STUDY 

In discussing the most feasible data capable of 
presentation in this particular study, it is necessary to 
consider the constraints faced by the evaluators. 

The first thing that must be noted is that this 
cost-effectiveness analysis was commissioned in July, 1975, 
after the school year under evaluation had already been 
concluded. This precluded the collection of certain tjrpes 
of data and m.andated that the cost-effectiveness team 
perform their analysis based upon the design that had been 
implemented during that prior year. 

It should be noted that in part because of the 
concerns of school districts and by the conception of the 
program by those involved, there were no control groups 
established. Thus, there are no comparison programs in the 
school districts for v^hich comparable outcome data are 
available. There may be other programs within some of the 
districts which provide compensatory education services but 
the nature of the selection of students for those programs 
and the co-mingling of funds from different sources, make 
comparisons next to impossible for this study. More 
thoughtful attention to potential comparison programs at the 
outset of the second year evaluation might lead to potential 
cost-effectiveness comparisons for the school year 1975-1976. ' 



A second determination at an early stage of the 
implementation of the program also hindered the cost-effective 
ness analysis. It was determined early in the school year 
1974-1975, that undue burdens of test collection \^ould not 
be imposed upon the schoo*^ rli sin--' - ts Thus, the students in 
all school districts ols would not be require*^^ 

to take all tests. I .cheme was designed whereby 

different tests would be taken by different students. This 
was perfectly appropriate methodology for providing outcome 
information for the evaluation of the ALL WIN-U.S.A. program. 
It does, however, hamper the cost-effectiveness analyses 
that might be performed. Furthermore, the funding of the 
cost-effectiveness analysis at a late date did not allow for 
the collection of fiscal data during the course of the 
school year. Thus, we were unable to systematically record, 
for example, the amount of time that principals spent on the 
program or the regular supplies and materials utilized 
within the program. These data had to be collected from 
school district personnel during July and August on a best 
estimate basis using a questionnaire that we devised for 
extracting that information. 

A further confounding factor was the difficulty of 
obtaining data from school district personnel during the 
months of July and August when many of them are typically on 
vacation. Enough said. 

The cost-effectiveness evaluation team within all of 
the constraints listed above nonetheless recognizes that the 
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function of an evaluation is to provide the best information 
possible for decision making. While the researcher's 
preference might sometimes be to throve up his hands in dismay 
and say "forget it", the evaluator will nonetheless attempt 
to provide vjhatever best information is possible, knov^ing 
that a decision must be made. That stance adequately 
describes the position of the cosi:-e .lectiveness evaluation 
team of this study. 

DESIGN FOR COST-EFFECTIVENESS STUDY 

This cost-effectiveness study must, of necessity, 
examine cost relationship of the program involved; outcomes 
(or effectiveness) must also be considered and, finally, 
the relationship between cost and the outcomes must be 
considered in some appropriate manner. 

Fi seal data for this study v?ere collected from the 
various school districts in order to determine those program 
costs that had been paid from district funds. EEA staff met 
with representatives of the various school districts in order 
to determine the appropriate fiscal data that might be 
available and the most feasible manner of obtaining the data. 
Next, a questionnaire was sent to each school district 
requesting various fiscal information, projections of time 
spent on the program by related personnel and district fiscal 
policy relevant areas. Based upon these data (as modified or 
refined by follow-up telephone calls) various calculations- 
were performed in order to determine district incurred 
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financial expenses. The procedures and formulae for cost 
determination are detailed in Chapter II of this report. 
Additionally, costs paid for by Los Angeles County Schools 
out of the OEO contract were carefully examined in order to 
determine the manner in which to best prorate them. These 
costs determinations are also detailed in Chapter Two. 

Outcome data collected as a part of the initial 
ALL-WIN evaluat- ^"^pleted August-t5-, were utilized for 

considering th " -iveness of the program. Data from 
various of the outcome determinations are summarized in this 
report but unique analyses have been performed; only for three 
of the cognitive achievement measures. For purposes of this 
study, it was necessary to have both pre- and post-test data 
from the same schools. This kind of data was only available 
for the standardized test in reading and the standardized 
test in mathematics. Thus we have confined our new analyses 
to these two measures. 

Difficulties in the analysis . occurred in the attempt 
at drav7ing comparisons between fiscal data of the ALL WIN 
schools and fiscal data for "comparable programs." Similar 
difficulties occurred in making comparisons of ALL WIN 
effectiveness data with that of other programs. Stated 
simply, and as previously noted in the constraint section., 
the lack of comparison programs within the implemented 
districts offered great difficulties. The study team, 
however, has attempted to overcome the difficulties by 
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getting '^normative" data of compensatory education student 
achievement and compensatory education costs based upon an 
extensive computer search of the literature. This computer 
search, v?hich is detailed more extensively in Chapter IV, 
yielded 681 references which were systematically examined by 
the evaluation staff in a lengthy period of library research. 

Moreover, contacts were made viith a number of 
individuals to determine whether they were aware of appro- 
priate ri-' -. aier programs that lu. -ht be used for 
''normecc v.: ' pUi_-poses. Dr. Alex Law and Dr. Vince Madden of the 
Office of Planning and Evaluation, California State . Department 
of Education, were contacted and EEA V7as allowed to purchase 
a computer print-out of the test scores of all compensatory 
education programs in California. Dr. Madden indicated that 
the California State Department of Education no longer 
collects fiscal data on these programs aey did several 
years ago) as the aniount of "noise" in - data was far more 
than cculd be tolerated. In short, the . didn't know v^hat they 
were eaziting. 

Dr. George Mayeske of the Office jf Planning and 
Evaluation (USOE) V7as contacted and indicated that he was not 
aware of any studies that would provide the kind of data 
needec for this investigation. He indicated that there were 
two studies being condnxzted by their of3.ce that will not be 
cozapleted until next January. Ms. Sue Baggart xv'as contacted 
and TSS- unable to offer suggested references but offered 
multipX^ caveats about the dangers of using fiscal and 
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outcome data from other projects for normative purposes. 
We are av?are of such dangers and have approached this task 
\Q±th considerable caution. While comparisons of ALL WIN cost 
and effectiveness data have been made with cost and effective- 
ness data from other projects, we nonetheless, have attempted 
to point out the deficiencies of . such comparisons and the 
dangers i.xilaerent therein. Contacts were also made with 
Dr. Charles Blashchke to discuss the cost-effectiveness 
evaluation performed by Education Turnkey for the Michigan 
State Department of Education, i^ome data v7ere obtained from 
that orgainization that proved to be worth further examination.) 
The data from that study are discussed more fully in 
Chapter IV. 

T sper'cific cosl:-ef f ectiveness ratios drawn in this 
study are. . .n essence, cost per gross outcome unit measures, 
as describes ' raldrip (1974). In essence, this amounts to 
the provisiirai / C data reflecting the incremental (and total) 
cost per nrcnt±i o£ student gsin in outcome score^ We have, 
to some e: v.tent: anrtempted to provide cost per refined unit 
data, baa... uipom the use of regression residuals, but such 
data is noL complete owimg to the constraints inherent in 
this study^ 
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COSTS OF THE ALL WIN-USA PROGRAM 
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One of the major objectives of the study was to 
calculate the total fiscal costs of the ALL WIN-USA Program 
for its first full year of operation. To achieve this, the 
evaluation staff first assembled a questionnaire designed 
to identify the costs incurred by each district that par- 
ticipated in the program (see Appendix 1 ) . In some instances 
data were requested which were intended to be used' by 
evaluation project staff in' performing further calculations. 
Next, in order to calculate the costs of the program that 
were paid directly by the grant itself, the staff visited 
the Of fLce of the Los Angsles County Superintendent of 
Schools and obtained the data from the Accountant and 
Special Projects Unit of the Internal Business Management 
Division. These combined .expenditures-- those made by the 
participating districts and those made by the grant-repre- 
sented the total costs of the program for 1974-1975. 

The Questionnaire 

Most of the items on the questionnaire asked respon- 
dents either to supply specific informatioxi from their 
respective districts 1974-75 budgets or to estimsite the 
amount of time various personnel spent in the program. 
Another item disked respondents to estimate approximately 
hov7 much was spent on regulHir district supplies and materials 
for students enrolled in the program. Several items dealt 
wiith the areas of fringe benefits and one additional item 
fccxised on whether districts were leasing space for the 
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instructional laboratories which were the integral part of 
the program. 

The questionnaire was mailed in early July, 1975 to 
each of the eight project directors. A cover letter was 
enclosed that indicated the purpose of the questionnaire and 
specified where to contact the evaluation staff if they had 
any questions (see Appendix 1) . Tr^Tephon ' ' i so were 
made to ensure that the questionnaire had been received as 
well as to emphasize tke importance of completing it by the 
deaidlrne of July 28. 

Upon return of: -the first questionnaire, it was 
discovered that a prx>ject director had misunderstood three 
items.. He had interpreted item three on the questionnaire 
to meaTi the major expenditures for special supplies and 
materials paid by the grant--instead of the more modest 
on^ rarade by the distxict for its regular supplies and 
materials which were consumed by students during the course 
of the program. (The expenditures for special supplies-- 
since they wex'e supported by the grant--were obtained from 
the Office of the Los- Angeles County Superintendent of 
Schools.) Items four and five_.were potentially intcrpret- 
able as meaning the expenditures made by the district for 
the operations and maintenance of only the ALL WIN-USE 
Program- -instead of the district's expenditures for operations 
and maintenance of its entire school program--as reported in 
its 1974-75 budget. As a result of these misinterpretations, 
the evaluat±on staff telephoned each of the project directors 
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to clarify what was meant by these three items. This extra 
time spent in telephoning each director was worthwhile, 
since it became evident that several of the other directors 
might have interpreted these items incorrectly. A more 
extensive prior field test of the questionna would ' ..ve 
been desirable, but s^iice thr e:itira term of this evaluation 
project was limited, that was not possible. Initial inter- 
views with several project directzors and close contact with 
all directors during the cOiirse of the data collection 
alleviated potential problems. 

Throughout the month of July, several districts were 
once again telephoned to encourage the directors to complete 
their questionnaires. (July was not, of course, an ideal 
time to collect data, since several of the directors were 
about to leave on vacation, others had just returned, and 
one had already left.) However, by the first week in 
August, all questionnaires had ieen returned and were ready 
to be analyzed, 

DISTRICTS' EXPENDITURES FOR THE ALL-WIN USA PROGRAJl 

Expenditures for Salary 

Tables II-1/II-2 illustrate the amount of salary 
that was charged against the ALL WIN-USA Program for the 
administrative staff in each district in the program during 
1974-75. T^^o types of administrators were involved: (1) 
projecr: directors who were located in each district's central 
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TABLE 11-1 



Dis!:rict Administrators' Salaries Charced to ALL WIN-USA 
Program by District for Fiscal Year 1974-75 



School District Numbc ^; c 

in Program 



rators Salar} x^ercent of Aniount of Salary 
FY 19 ?5 Time in Pro- Applicable to 



(1) 


(2) 


(3) 


gram 


Program 
(5) 


Colton 




• $28,849 


10.0 


$2,885 


Compton 






1.0 


276 


Duarte 




.27,915 


5.0 


1,396 


Fontana 




22,420 


2.0 


448 


Garvey 




25,000 


8.0 


2,000 


Lawndale 




22,568 


7.0 


1,580 


L.A. County Spec. Sch. 




43,020 


5.0 


2,150 


Ontario-Mont clair 




27,000 


3.5 


. 945 






19,612 


1.2 


235 



Sources: Column 2 and Column 3: Data supplied by participating districts. Column 
A: Estimates made by participating districts. Column 5: Column 3 x Column 4. 
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TABLE II-2 



Principals' Salaries Charged to ALL WIN-^USA 
Program by District for Fiscal Year 1974--75 



School District 


Number of 
Principals 
in Program 


Salary 
FY 1975 


Percent 
Time in 
gram 


of 

Pro- 


Amount of S 
Applicable 


(1) 


(2) 


(3) 


(4) 




(5) 


Pol ^nn 


A 

^ 


9 88,647 


25. 0 




$ 22,162 


L'OiupLon 


S 


200,813 


1.0 




. 2,008 


Duarte t 


3 


71,720 


5.0 




3,586 


Font ana 


8 


178,295 


2.5 




4,457 


Garvey 

1 


1 
1 


23,444 
23,976 


15.0 
8.0 




3,517 
1,918 


LaTOdale 


5 


110,059 


3.0 




3,302 


L.A. Counry Spec. 


Sch. 5 


105,910 


5.0 




5,293 


Ontario-Mont clair 


2 
2 
1 
1 
1 
1 


45,321 
46,545 
21,507 
22,865 
23,879 
21,252 


2.3 
2.1 
1.4 
2.5 
4.0 
3.4 




1,042 
977 
301 
572 
955 
723 



Sources: Column 2 and Column 3: Data supplied by participating districts. 
Column 4: Estimates made by participating districts. Column 5: Column 3 
X Column 4. 
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offices and (2) building principals at each participating 
school. 

While the Los Angeles County Special Schools and 
the Ontario-Montclair District assigned two individuals in 
their central offices responsibilities for administering 
the ALL WIN Program, the remaining districts assigned only 
one person to assume the role of project director (see 
Table Il-i) . However, for both Los Angeles and Ontario- 
Montclair, the combined time spent by both administrators in 
the program (10.0 percent and 4.7 percent respectively) in 
most cases was not reported to be much more than the time 
spent 5.n the districts with one administrator. Compton 
and Fontana's central administrators reported spending the 
least time in the program (1.0 percent and 2.0 percent 
respectively) . It is important to keep in mind that the 
percent of time in the program was estimated by each project 
director when he completed the questionnaire and was not done 
through direct observation by the evaluation staff. This 

fact may explain some of the variation among the districts' 

u 

estimates. We would recommend funding this portion of the 
evaluation in the early part of the school year hereafter 
in order to enable the evaluators to collect observation 
data on this dimension. 

As in Table II-l, Table II-2 shov7S a fairly V7ide 
variation in the time spent. (Again, these V7ere estimated 
times~-and not arrived at through direct observation,) The 
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four principals in Colton spent considerably more time^ than 
their counterparts in the other districts (25.0) and those 
in Compton spent the least amount of time (1.0). In most 
cases the project directors who completed the questionnaire 
indicated that the principals in each of their respective 
districts spent the same amount of time in the program. In 
contrast, both Garvey and Ontario-Montclair indicated that 
the principals in their districts spent varying amounts of 
time in the program. 

Table II-3 illustrates the number of teachers who 
participated in the ALL WIN Program during 1974-75 and their 
annual salary charged against the program. Since all the 
instructors were full-time faculty members in the program, 
one hundred percent of their salaries were charged to the 
program. The Los Angeles County Special Schools had the 
most teachers in the program (10) and Duarte and Garvey had 
the least amount (3 each) . 

Table II-4 shows the number of instructional aides 
who participated in the program as well as their annual 
salary charged against the program for 1974-75. As was the 
case with the faculty members, all the instructional aides 
were full time in the program. In most of the districts, 
there was at least one instructional aide with each full- 
time teacher. The exceptions were the Compton District 
with eight teachers and seven instructional aides and the 
Los Angeles County Special Schools with ten teachers 'and 
two instructional aides. Since the types of schools that 
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TABLE II-3 



Teachers' Salaries Charged to 
ALL VIN-USA Program by District 
for Fiscal Year 1^74-75 



School District 



(1) 



Number of 
Teachers in 
Program 

(2) 



Salary 
FY 1975 



(3) 



Percent of 
Time in 
Program 

(4) 



Amount of Salary 
Applicable to 
Program 

(5) 



Colton 


4 


$ 58,597 


100 


$ 58,597 


Compton 


8 


119,891 


100 


119,891 


Duarte 


3 


36,950 


100 


36,950 


Fontana 


8 


112,855 


100 


112,855 


Garvey 


3 


43,116 


100 


43,116 


Lavmdale 


5 


71,008 


100 . 


71,008 


L.A- County Spec. Sch. 


10 


141,110 


100 


141^110 


Ontario-Montclair 


8 


122,430 


100 


122,430 



Sources: Column 2, Column 3, and Column 4: Data supplied by participating 
districts. Column 5: Column 3 x Column 4. 
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TABLE II -4 

Instructional Aides* Salaries Charged to ^LL WIN-USA 
Program by District for Fiscal Year 197A-75 



School District Number of Aides Salary Percent of Amount of Salary 

in Program FY 1975 Time in Applicable to Program 

Fiogram 

. (1) (2) (3) (A) (5) 



Colton 


6 


$16,683 


100 


$ 16,683 


Compton 


7 


2A,522 


100 


24,522 


Duarte 


3 


15,990 


100 


15,990 


Fontana 


9 


32,594 


100 


32,594 


Garvey 


3 


13,181 


100 


13,181 


Lawndale 


6 


16,549 


100 


16,549 


L.A. County Spec.Sch. 


2 


10,587 


100 


10,587 


Ontario-MontcJ.air 


8 


30,248 


100 


30,248 



Sources: Column 2, Column 3, and Column 4: Data supplied by participating 
districts. Column 5: Column 3 x Column 4. 
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are administered by the Los Angeles County Superintendent 
of Schools are unique (schools designed to enroll students 
classified as delinquents) , it is not surprising that the 
data for these schools do not always follow the same pattern 
as those for the other districts. In this instance, there 
were fewer aides for the program, in all likelihood, because 
each teacher in the program had responsibility for fewer 
students than the teachers in the other districts (see Table 
11-16) . The special nature of the schools dictates that 
class size be considerably smaller than it is in a more 
typical school district. 

One should also mention that even though all the 
instructional aides spent full time in the program, some 
of their salaries were computed on an hourly basis, some 
on a monthly basis, and others on an annual basis. 

Table II-5 illustrates the estimated expenditures 
for substitute teachers who replaced the regular full-time 
teachers in the program during 1974-75. The expenditures 
are estimated, since the project directors were not asked 
to identify the number of actual days each teacher in the 
program was absent during 1974-75. Instead,^ the evaluation 
staff estimated seven days of leave for each instructor. 
This estimate was based upon the staff's experiences with 
districts throughout the state. (The California Education 
Code specifies ten days of leave per iDStructor, but the 
staff felt this did not properly reflect the actual number 
of days leave taken.) Thus, the total Dumber of days absent 
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was arrived at by multiplying the nimber of teachers in the 
program in each district by the niomber seven. This product 
was then multiplied by the rate of pay for substitutes-- 
provided by the project di "rectors through :zh^ questi onnairB 
The r = -t is the 3d expenditures fcr substitute 

teach'^-rrr^ i.n the ALL V/IK ' rogram by . distr-i -t . It should be 
poin'^ed rat that these :are the only datr n xhe study, which 
were erlved through estimates made by "l... e-^aluation s^aff 
ratheir tinan by the project directors in zh disirrlct. 

The total amoum of salary for 7 ^sonnel in the 
ALL WIl Program--calcuiated from Tables II-l-5--is .shown in 
Column 2 of Table 11-15. 

Expenditures for Health-Welfare Benefits 

The project directors were asked to supply the 
evaluation staff with the health-welfare benefits that were 
given by each district to its certificated employees as well 
as to its classified employees. These data are shown in 
Tables II-6-8 and Table II-9 respectively. 

Tables II-6-7 depict the expenditures for health- 
welfare benefits for administrators (project directors and 
building principals) who had some responsibilities in the 
ALL WIN Program during 1974-75. The annual expenditures 
for these fringe benef its--supplied by the project directors- 
were multiplied by the percent of time each administrator 
spent in the program. Although the fringe benefit contri- 
bution per certificated employee in Compton is considerably 
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hig:Tii*r than those of other dlsirricts — witih zhe exception 

of zhe Lgvs Angeles Coirr_ Sped aJL 3chools--the project 
d::.r3Ctor and four builiii g prinrri '.:)als spent only c i.e per- 
ZBTTZ of tiieir work loads in t3£ ^.rogram. The re. :. , of 
conrrse, i3 a mod'HSt: exr::-:nditi:rr=: :o:': health and are 
charged ^^igainst tihe prcgirairi, HDvr-tSver, since the -:ir~itri- 
but^irn fox health and wrelfare in r±ie Los .Angele:s i^oimity 
Special Schools is high — and r±ie percent of time it;: ad- 
min:rs tirators are spending in zhe program is close to the 
mean of all districts (4.7 and 5.7 respectively' — tiia result 
,is the highest expenditure for health and -welfarB in all 
districts , 

Table II-8 illustrates the districts' e:rpeiidf.tures 
for health, and vw-elfare benefits for teachers in zh:^ ALL V7IN 
Prognram. Since all ins true torrs were full time in tfc^ program, 
one hundred percent of the average annual expenditi^re per 
certified employee *'7as charged against the prograir As in 
Tables II-6 and II-7, the Compton District and tr.e Lcs 
Angeles County Special Schools nad the highest expenditur£3 
for health and welfare benefits 

Table II- 9 shows the expenditures that districts 
contiributed towarc: health and welfare benefits fcr 
instructional aires in rhe ALL 'ELN Program. As in Table 
II- 8 , the averagre expemfiture classified employee was 

charrgco against the prcr^grHm, since .all ins trriicti-anal aides 
were: furl-"., time in the ^Erx^gram. In three d:}.srxicT:s , the 
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expenditures per classified eaiployee were the same as 
those for certificated employees. Ir: Ontario-Montclair 
the expenditure per classified empJLoT^e vas repctrted. as 
higher than that per certificated employee. As in Tables 
11-6, 11- 7, and II- 8, tne expenditiireis for health and 
welfare per certificated employee ar^: highest in the 
Camp ton District and the Los Angeles ::otinty Special Schools. 
Vith the exception of L^s Angeles Comty Special Schools, 
all of the ins true tioniii aides ~s^7ithi~ each district ^irere 
given the same expendir:~e for 5ieaLtk and waliare. In the 
special schools of Los Angeles County ^ however, each of the 
two instructional aiis;f. -^a^ givem dir-'^^'^azrent allocurricms, 
since one was employed by the Count:* a fuil-tinnsL bt^sis 
and the other on only a 75 paxcerr^t basis. 

Expenditures ror -Re uiremti'-r ^lanexit:. 

The es^enditmres ^-"jr re~i:rei3Hnt hene^irrs . -i^ra 
computed for hoth cerrtif :. .ared c^nd c:^^'issif ied eu:z_oyees of 
the ALL WIN Program. Hiir:-^^ are ihcvvn in Tabl=:3 I : -10 and 
11-11. Since all of the .rnaployaes in rhe study v-rire part 
of either the California State TeackexH Retirement. System 
or the California Public IZmployes^s P^etirement 3ysr:^in, there 
was no need to collect xr '^ireiznein:: cLrr-i i^roin thiv project 
directors. Table Ii;^:;L^ siiov7S thar ^ ^ pexcent or the 
salaries for certificated erap]k)ye£.v. .nentral arri.nis trators , 
building principals, and iinis trucziorrs^ in the p ro:zr:am was 
the retirement contribution each disnrict made; and this 
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amoiant was charged against the ALL V7IN Program. In turn, 
Tab~.e 11-11 shows that 13,45 percent of the salaries for 
classified employees (instructional aides) in the program 
was contributed for retirement, and this was also charged 
against the program. One should mention that 5.85 percent 
of the 13,45 percent for classified employees was the 
distxict contribution toward Social Security benefits and 
the remaining percent went to the California Public 
Employees Retirement System. 

The total amount of health and welfare benefits 
Including retirement benefits for personnel in the ALL WIN 
Trcg;ram--calculated from Tables II-6 to II-ll--is shoxm in 
ColLjnn 3 of Table 11-15. 

Expnnditures for Regu lar Supplies and Materials 

Table 11-12 illustrates the expenditures for regular 
supplies and materials that were consumed by students in 
each district who v/ere enrolled in the ALL WIN Program during 
1974-75. These expenditures were estimated by the project 
directors when they completed their questionnaires. It must 
be stressed that these are supplies and materials other than 
those which were paid for directly by the grant itself. This 
latter expenditure- -in comparison to the districts' expendi-- 
tures for the more or less daily supplies and materials 
were far more extensive. 

The total expenditures for regular supplies and 
materials in all districts is shovm in Column 4 of Table II--15 
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TABLE 11-10 



Expenditures for Retirement Benefits for Certificated Employees 
in ALL WIN-USA Program by District for Fiscal Year 1974-75 



School Dir-trict 



(1) 



Salaries of Certifi- 
cated Employees Appli- 
cable to the Program 

(2) 



Percent of Employee 
Salary Contributed 
by District for 
Retirement 
(3) 



Cost of Retirement 
Contribution Appli- 
cable to Program 

(4) 



Colton 


$ 83,644 


4.8 


$4,015 


Compton 


122,175 


4.8 


5,864 


Duarte 


41,932 


4.8 


2,013 


Fontana 


117,760 


4.8 


5,652 


Garvey 
« 


50,551 


4.8 


2,426 


Lawndale 


75 , 896 


4.8 


3,643 


L.A. Co. Spec. Sch. 


148,553 


4.8 


7,131 


Ontario-Mont Clair 


128,180 


4.8 


6,153 



Sources: Coluirai 2: From Table II-1/II-3. Column 3: California Education 
Column 4: Column 2 x Column 3. Code. 
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TABLE 11-11 

Expenditures for Retirement Benefits for Classified Employees 
. in ALL IVIN-USA Progra^Ti by District for Fiscal Year 1974-75 



School District Salaries of Classified Percent of Employee 
Employees Applicable Salary Contributed 
to Program by District for 

Retirement 



(Ij 


(2) 


(3) 


(4) 


Co It on 


$16,683 


13.45 


$2,244 


Compton 


24,522 


13.45 


3,298 


Duarte 


15,990 


13.45 


2,151 


Fontana 


32,594 


13.45 


4,384 


Garvey 


23,181 


13.45 


1,773 


La\^dale 


16,549 


13.45 


2,226 


L.A. Co. Spec. Sch. 


10,587 


15.45 


1,424 


Ontario- Mont clair 


30,248 


13.45 


4,068 



Cost of Retirement 
Contribution Appli- 
cable to Program 



Sources: Column*2: From Table II-4 • Column 3: California Education Code* 
Coluimi 4: Column 2 x Colimn 3, 
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TABLE ir-12 



Expenditures for Regular Supplies and Materials Used in ALL 
WIN-USA Program by District for Fiscal Year 1974-75 



School District Regular Supplies and Materials 

* (1) (2) 



Colton 


$ ISO 


Compton 


3,674 


Duarte 


3,888 


Font an a 


3,016 


Garvey 


5,420 


Lawndale 


100 


Los Angeles County Special Schools 


1,780 


Ontario-Montclair 


1,400 



Sources: Column 2: Estimates made by participating districts. 
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Expenditures for Plant Operations and Maintenance 

Tables 11-13 and II-'14 illustrate the percent of 
expenditures by district for plant operations and maintenance 
that were charged against the ALL WIN Program. The project 
directors were asked to provide the total expenditures 
as recorded in their districts' 1974--75 budgets--for plant 
'Operations and maintenance as well as the total number of 
full-time teachers in their districts. The evaluation 
staff then computed the cost of both plant operations and 
maintenance per full-time instructor in the district and 
multiplied each of these amounts by the number of instructors 
in each district who participated in the ALL WIN Program. 
Garvey and the Los Angeles County Special Schools had the 
lowest total expenditures for plant operations (the latter 
schools, in all likelihood, because of their special nature 
and the fact that some maintenance and operations costs might 
be subsumed in an other than education budget), and Compton 
and Colton had the highest total expenditures. For main- 
tenance, Garvey, Duarte, and the Los Angeles County Special 
Schools had the lowest tol:al expenditures and Compton and 
Onta;rio-Montclair had the highest. 

The total expenditures for plant operations and 
maintenance in all districts is sho^vn in Columns 5 and 6 
respectively of Table 11-15. 
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The total district expenditures for the ALL WIN 
Program are shown in Column 7 of Table 11-15. These 
expenditures were the direct costs of instruction for the 
program during 1974-75. In terms of total expenditures, 
Duarte and Garvey had the lowest expenditures and Compton, 
.Fontana, and the -Los -Angeles Special Schools had the highest 
expenditures. The total direct expenditures in all dis- 
tricts for the program were $1,249,288. 

Table 11-16 presents both the number of stations 
(booths that contain the New Century audio-visual devices) 
that V7ere available in each district as well as the number 
of students who were enrolled in the program. T\rhile the 
official enrollment by center was reported monthly to the 
Office of the Los Angeles County Superintendent of Schools, 
the enrollment shovm in Table 11-16 was taken from the June 
1975 report. The evaluation staff compared these data with 
^those from other monthly reports and found that the anroll- 
ment figures did not vary significantly among the reports. 

Column 5 of Table 11-16 shows the total district 
costs per student of the program. This per student cost 
was derived by dividing the total costs for each district 
by the district's enrollment. The average per student cost 
was $156. Duarte had the lowest per student costs of all 
districts and the Los Angeles County Special Schools had 
the highest per student costs. The costs were high for 
the special schools of Los Angeles County because each school 
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in the program Irad a lower teacher-s tudent: ratio than did 
the other districts that participated in the program. 
Another possible explanation for the cost of the program 
in special schools is the extent to which it was used as 
the "regular'' program rather than a "supplementary" one. 

EXPENDITURES FOR THE ALL. WIN- USA PROGRAM 
MADE FROM THE FEDERAL GRANT 

In addition to the costs incurred directly by school 
districts, there were other expenses paid for dirrectly by 
trie federal grant. TSvaluation project sitaff feels that it 
is: iii30rtant to exanine these costs in two caitegcries: on- 
going expenditures , and start-up expenditures. 

On~Going Expenditures 

Table 11-17 illustrates the expenditures that were 
considered to be on-going each year that the program is 
operated. These expenditures were paid for directly by the 
grant. The largest on-going expenditure was for instructional 
supplies. This amount is considerably higher than the ex- 
penditures made by the individual districts for their 
regular supplies and materials (see Table TI-12) . The 
expenditures for th^ Los Angeles County Educa.tiional Auditoi^ 
and for Evaluation were grouped separately from the remain- 
ing on-going expenditures because they are both items that 
might undergo some change in scope each year that the 
program is conducted j.n school districts. The on-going 

11-27 
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expenditures excluding the two items relating to evaluation 
are $662,174. 

Start-up Expenditures 

Table 11-18 represents the start-up expenditures 
that were paid for directly by the grant. They were con- 
sidered to be *'start-up" because they would not have to be 
made <each time the program was offered in a school district. 
Each af the four items was amortized over a specified number 
of yeairs . This period of time was arrived at through 
esciinates made by eval--q tion project staff after consulta- 
tioxL iisdrth several project directors. 

The item of pra-service training involved a one- 
week txraining session of the ALL WIN .Erogram instructors 
prior to the initiation of the program. The rationalization 
for amortizing the session over a five-year period was that 
the instructors presently in* the program would, in all 
likelihood, not require any additional pre-service training 
in the first five years of the program. However, after 
five years there is a strong probability that the program 
would have to undergo some extensive changes which might 
then r^uixe new pre-service training. 

The audiovisual equipment and the typewriter (the 
latter purchased for the project director's secretary and 
housed at the Office of the Los Angeles County Superinten- 
dent of Schools) were amortized respectively over a four- 
and five-year period. The carrels, however, were amortized 
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TABLE 11-17 

t 

On-Going Expenditures of ALL WIN-USA Program 
Paid Directly by the Grant for Fiscal Year 1974-75 



Amount of Expenditure 
(2) 



Project Director . $ 26,232 

Typist 7,908 

Fringe Benefits 5,623 

In-Service Training 25,300 

Instructional Supplies 558,777 

Office Supplies 965 

Mileage and Conferences 2,169 

Contract Maintenance 35,200 

Sub-Total On-Going Expenditures $662,174 

Los Angeles County Educational Auditor 13,134 

Evaluation ^^>"^^ 

Sub-Total On-Going Evaluation Expenditures $ 71,210 

Total On-Going Expenditures Including 

Evaluation $733,584 



Sources: Column 1 and Column 2: Data supplied by the Office of the 
Los Angeles County Superintendent of Schools. 



Category 

(ID 
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TABIE 11-13 

Amortizaticn of Start-Up ExpGnditiirrcs of ALL WIN-USA P.^gram 
Paid Directly by the Grant ftrr Fiscal Year 1974-77" 



Category Amount of Start- Years of Ainnrtized 

•Up Expenditure Amortization E:3zpenditure 



CD 


(2) 


C3) 


(4) 


Pre-Service Training 


$ 70,400 


5 


514,080 


Audiovisual Equipment 


253,394 


4 


63,349 


Carrels 


54,875 


10 


5,487 


Typewriter 


770 


5 


154 


Total Amortized Expenditures 


83,070 



Sources: Columns 1 and 2. Data supplied by the Efface of the Los Angeles 
County Superintendent of Schools. Column Sz Estimates made by parricipating 
districts. Column A: Column 2 divided by Colnmn 3. 
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over a ten-year period, since unlike the equipment, they 
had no movable parts and therefore were not subjected to 
"wear ..and tear" by the students. The total yearly start- 
up expenditures, amortized over their respective periods, 
were $83,070. 

Total Grant Expenditures 

Table 11-19 illustrates the total expenditures 
(on-going and start-up) for the ALL WIN Program which were 
paid for directly by the grant. The total expenditures 
were $816,454. \Ttien this amount was divided by the total 
number of students enrolled in the program (8,002), the 
result; showed a total cost of $102 per student- -which was 
paid directly by the grant. 

TOTAL EXPENDITURES FOR ALL WIN- USA STUDENTS 

1'otal Direct Expendi tures 
(Not "Including Indirect Costs) 

Table 11-20 shows in Column 3 the total expenditures 
of the ALL WIN Program. These expendj' tnrr-s represent the 
total expenditures made by all the districts that partici- 
pated in the program (Column 1) as well as those expenditures 
that were paid for directly by the grant (Column 2) . Since 
the total expenditures for the program were $2,053,072, and 
there were 8,002 students enrolled in the program throughout 
the eight districts, the total cost of enrolling each student 
in the program for 1974-75 was $258. 
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TABLE 11-19 



Total Grant Expenditures Per Student in ALL WIN-USA 
Program for Fiscal Year 1974-7S 



Total On-Going 
Expenditures 
Including 
Evaluation 



Total Amortized 
Expenditures 



Total Costs 
Paid by Grant 



Number of 

Daily 

Students 



Total Costs 
Per Student 
by Grant 



Paid 



CD 



(2) 



(3) 



(4) 



C53 



$733,384 



$83,070 



$816,454 



8,002 



$102 



Sources: Column 1: From Table II-17 • Column 2: From Table 11-18. Column 
3: Total of columns 1 and 2. Column 4: From Table 11-16. Column 
5: Column 3 divided by Column 4. - 
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Total District Expenditures 
(Including Other Program costs) 

Table 11-21 illustrates the total of other program 
expenditures incurred by each district that participated in 
the program. These expenditures were calculated in the 
following way. The eight project directors indicated on 
the questionnaire the total amount of expenditures that 
were made in each of their districts for 1974-75. The 
evaluation staff subtracted from each of these totals the 
direct cost of the ALL WIN Program in each respective 
district (see Table 11-15). These new totals (Column 4 
of Table 11-21) then represented each district's expendi- 
tures excluding their expenditures for the ALL WIN Program. 
By dividing these totals by each district's average daily 
attendance for 1974-75 (which the evaluation staff obtained 
by telephoning the offices of the Los Angeles County 
Superintendent of Schools and the San Bernardino County 
Superintendent of Schools), the total district costs per 
student excluding the costs of the ALL WIN Program (Column 
6 of Table 11-21) V7ere derived. By multiplying each of 
these costs by the number of students enrolled in the ALL 
WIN Program in each district, the evaluation staff obtained 
the costs of the regular program in the eight participating 
districts (Column 8 of Table 11-21). 

Compton had the highest indirect costs of the 
program and Los Angeles County Special Schools had the 
lowest regular program costs. Since Compton had the highest 
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total expenditures^ and Los Angeles County Special Schools 
had one of the lov/est of the eight districts, these results 
are not surprising. Because of the special nature of the 
schools administered by Los Angeles County, the evaluation 
staff calculated the total regular program costs of the 
program in each district as well as the total regular pro- 
gram costs in each district except the Los Angeles County 
Special Schools. Each of these totals are also shown in 
Table 11-21. 

Table 11-22 depicts the total expenditures includ- 
ing the regular program costs that were Incurred during 
1974-75 in conducting the ALL WIN Program (Column 5) . When 
these total expenditures were divided by the total number 
of enrolled students, the total expanditures per student 
C$1,380) were derived. 

Once again, because of the special nature of the 
schools administered in Los Angeles County, the evaluation 
staff also calculated the total expenditures of the ALL WIN 
Program in all districts except Los Angeles County. To do 
this, the staff subtracted the total expenditures for Los 
Angeles County Special Schools (sho\^7n in Table 15) from the 
total expenditures made by all districts. Tae staff also 
subtracted the. regular program costs for the SDecial schools 
of Los Angeles County (shown in Table 11-21) from the total 
regular program costs of the program in all districts. To 
subtract the share of the expenditures made by the grant for 
Los Angeles County Special Schools, the staff first determined 
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the percent of total students which were enrolled in the 
special schools (11.32 percent). This percentage was then 
multiplied by the total grant expenditures to obtain the 
grant expenditures that went to the special schools of Los 
Angeles County ($92,423). Subtracting that amount from the 
total grant expenditures produced the total expenditures 
made by the grant to all districts excluding Los Angeles 
County Special Schools (Column 4 of Table 11-22). Finally, 
dividing this amount by the ntmiber of students enrolled in 
all districts except Los Angeles County Special Schools, the 
evaluation staff derived the total expenditures per student 
(excluding Los Angeles County Special Schools) enrolled in 
the program (Column 7 of Table 11-22) . 
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The educational impact of the ALL UIN-USA Program 
during the 1974-75 school year has been assessed and re- 
ported in the final evaluation report of Educational 
Evaluation Associates. In an effort to identify information 
that will be most useful in perfonrang a rigorous cost- 
effectiveness analysis of the ALL WIN Program, we reviewed 
the evaluation procedures and outcome data that V7ere prev- 
iously reported. 

Although a number of different outcome measures 
covering a variety of program effects were utilized in the 
original evaluation design, only the student cognitive out- 
comes lend themselves to a reliable cost-effectiveness 
methodology. Measures of student, staff and parent affect 
were used in the evaluation, but the outcome data generated 
by these instruments must be cautiously regarded as 
"suggestive'' of the program's impact. This concern for the 
possible unreliability of these data was shared by the EEA 
evaluators as they commented that "the general ineffective- 
ness of affective measures or the inability of educational 
programs to altar affect is, of coarse, a hotly debated 
issue in the field of educational evaluation" (EEA; 1975, 
p. 55). 

Student cognitive effects were assessed with stan- 
dardized achievement measures and criterion-referenced 
tests. The latter were constructed to measure specif ically 
the learning objectives of the New Century educational 
program, the curriculum package which provided the ins true - 
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tioxial foundation for the ALL WIN-USA Program. The testing 
procedures for these criterion-referenced tests followed a 
rather complex sampling plan that was designed to avoid 
overlap with district standardized testing, and to minimize 
testing time in general. Tne actual' sampling plan for the 
criterion-referenced reading tests appears in Table III-l, 
while Table III- 2 presents the comparable plan for Mathematics 
tests. It is important to note from these tables that 
approximately half of the learning centers were scheduled 
for either pre- testing or post-testing. Unfortunately, 
collection of these outcome data revealed that only four 
mathematics centers reported complete pre- and post-test 
results, while comparable data for the reading centers was 
even more critically scarce. Since complete pre- and post- 
test data for the same schools are necessary for meaningful 
cost-effectiveness assessments , the criterion-referenced 
results are unusable for our purposes. Consequently, we 
defer further discussion of these data in favor of the 
standardized achievement outcomes. 

Procedures for the analysis of standardized achieve- 
ment outcomes included a pretes t-pos ttes t gain analysis, as 
well as a time series assessment of the program ' s impact . 
The latter was performed by comparing student performance 
on standardized tests to thair achievement growth in previous 
years. Records of ALL WIN students in grades three, four, 
five and six were searched for test scores from the years 
1973 and 1974. Stud -nLo with comparable end^of-year test 
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TABLE III-l 



School and Grade 


Sampl Ing Plan 


for Cri 


terlon- 


■Referenced 


Read 


Ing Tests 


D!:3trict 


School 






Grades 












5 


6 


7 


8 


9 10 


Duarte 

\Jl 1 1 1 1 


Andres-* 
Dua r te 

Duarte 
nign Scnool 






Pre 


Pre 


Pre 


Pre/ Pre/ 
Post Post 




Northview 




Post 


Post 


Pre 


Pre 




Lawndale 


Anderson 


Pre 


Pre 


Pre 








Elementary 


Roosevel t 
Addams 
Green 
Rogers 


Pre 

Post 

Pest 


Pre 

Pos-t 

Post 


Pre 

Post 

Post 


Post 


Pos 




Fontana 


Randal 1 
Pepper 


PrG 


Pre 


Pre 








Ontario-- 


Mission 


Pre 












Montclai r 
















Col ton 


Lincoln 


Post 












Un I f led 
















Number pretested 




200 


150 


200 


100 


100 


80 80 


Number posttested 




150 


150 


150 


120 


120 


80 80 
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TABLE II I- 2 



School and Grade Sampling Plan for Criterion-Referenced Mathematics Tests 



District School Grades 

A 5 6 7 8 9 10 



Fontana 


Oleander 


Pre/ 


Pre/ 


Pre/ 






Unified 




Post 


Post 


Post 








N. Tamarind 


Pre/ 


Pre/ 


Pre/ 










Post 


Post 


Post 






Ontario- 


Imperial 








Pre/ 


Pre/ 


Montclair 










Post 


Post 




DeAnza 








Pre/ 


Pre/ 












Post 


Post 


Number pretested 




110 


110 


110 


180 


180 V 


Number posttested 




110 


110 


no 


180 


ISO 
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scores were then selected for an analysis of the program's 
impact during the present year over achievement growth in 
previous years when students did net participate in ALL WIN 
learning centers. The range of standardized instruments, 
their forms and levels of administration during the three- 
year period appear in Table III-3. 

In order to determine the differential impact of 
the ALL WIN program, two periods of achievement grovjth were 
statistically compared for different "cohorts,'' or groups 
of students. Period '^A" referred to achievement gain from 
1973 to 1974, and was termed "predicted gain'' since it repre- 
sents the amount of growth that might be expected in the 
absence of the ALL WIN program. Period "B" referred to 
achievement gain from 1974 to 1975, and was termed ''actual 
gain" since it represented the effect of the ALL WIN program. 
Superior performance by ALL WIN students would be indicated 
by greater achievement gain during Period B than during Period 
A. This information is summarized in the upper portions of 
Table III-4. Specific comparisons of actual and predicted 
gainc v7orG carried- out for four separate cohorts of students 
whose progress through the ALL WIN program began at different 
times in their educational careers. These groups are pre- 
sented in the lov7er portion of Table III-4. 

The results of this analysis are summarized in Table 
III-5 for the reading centers, and Table Ill-^e for the Math 
centers. Inspection of these tables indicates that about 
28% of ail reading center comparisons shovjed superior achievement 
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TABLE III-3 



Standardized Achievement Tests Administered In Each 
District cjnd Grade for Each of Three Years 



District 

Col ton Unified 

Compton Unified 

Fontana Unified 
Garvey Elementary 



Ontarlo--Montc1alr 
Elementary 



Grade 


7^; Post*^-e<it 




/J rosttest 


3 


CPT~23A 


CPT-ZSA 


CPT-12A- 




CTBS-Q2 


CPT-23B 


CPT-23A 


5 


CTBS-Q2 


CTBS-(12 


CPT-23B 


6 


CTBS-Q2 


CTBS-C12 


CTBS-Q2 


3 


CTBS-SC 


CPT-12B 


CPT-12A 


i) 


CTBS-Sl 


CPT~23B 


CPf-23A 


5 


CTBS-S2 


CTBS-Ql 


CPT-233 


6 


CTBS-S2 


CTBS~Q2 


CTBS-Ql 


A 


CTBS-Sl 


CPT~23B 


CRT-23A 


5 


CTBS-Q2 


CTBS~Q2 


CPT-23A 


6 


CTBS-Q2 


CTBS-Q2 


■ CTBS-QI2 


3 


CPT-23B 


CPT-23A 


CPT-12A 


h 


CTBS-Q2 


CPT~23B 


CPT-23A 


5 


CTBS-Q2 


CTBS~Q2 


CPT-233 


3 


CTBS-Sl 


CPT-23A 


CPT-12A 


k 


CTBS-Sl 


CPT-'23B 


CPT-23A 


5-6 


CTBS-S2 


CTBS-Q2 


CPT-23B 
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TABLE 1II-4 



ACHIEVEl'lENT GAIN PERIODS AND STUDENT COHORTS 
CX»MPAREn IN "STANDARIZED TIME-SERIES" ANALYSIS 



Gain Period 


Time Soan 


All Win Status 




A (predicted) 
B C actual ) 


1973 post-test to 1974 pre-test 

1974 post-test to 1975 pre^test 


Not implemented 
Implemented 


Student Cohort 


1973 


1974 
Grade 2 
Grade 3 
Grade 4 
Grade 5 


Grade 3 
Grade 4 
Grade 5 
Grade 6 


1- 2-3 

2- 3-4 

4-5-6 


Grade 1 
Grade 2 
Grade 3 
Grade 4 
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gain during the period of ALL WIN implementation. For the 
Math centers, only one comparison out of a total of three 
proved significant. The tentative nature of the Math center 
results was pointed out in the evaluation report. 

Procedures for the standardized pre-tes t/post-tes t 
gain analysis included all learning center students V7ho were 
pre- and post-^tested with a normed reading or mathematics 
test for purposed other than zhe evalur^tion of ' the ALL WIN 
program. These purposes included statewide, district and 
Title I testing, and utilized the Cooperative Primary Test 
(CPT) , the Comprehensive Test of Basic Skills (CTBS) , and 
the Wide Range Achievement Test (WvAT) . The actual 
distribution of these tests across districts and grade 
levels appears in Table III-7. 

Standardized achievement effects were determined 
by statistically comparing mean pre-test/post-tes t gain 
scores to an estimated criterion of 1.25 months growth for 
each month of center operation. Because ALL WIN centers 
were only operational for eight school months, the observed 
pre I" f^rs t , post^*~test g?^?!.ii55 v?ere Hujustc^d bv a f Victor of 
10/8 ths. This correction factor was intended to project 
each center *s achievement growth through a regular ten month 
school year. The. comparisons of "adjusted'* pre-tcs t/pos t-- 
test grov7th were then made for each school, district and 
grade level in both reading and mathem.atics . The results 
for each grade level are summarized in the upper portion of 
Table III- 8 for reading, and the lower portion of Table III- 8 
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for Math. Inspection of these findings indicates a greater 
achievement growth at upper grade levels in both reading and 
mathematics . 

In an effort to utilize the most powerful analytic 
tools to provide outcome data that would be useful for a 
cost- effectiveness analysis of the ALL WIN program, the 
'Standardized '-pre- and post-test scores were re-analyzed by 
the method of linear regression (Guilford, 1965). This 
procedure generates a statistical prediction equation that 
specifies the expected relationship between pre-test score 
levels and their corresponding predicted post-test levels. 
The analyses were perfonned separately for each grade level 
in reading and mathematics, and the results of these 
analyses .are presented in Table III-9 for the standardized 
reading outcomes, and in Table III-IO for the corresponding 
mathematics outcomes. Inspection of the prediction equations 
in Table III~9 indicates a differential pattern of pre-test/ 
post- test growth from grades 2 through 6. ¥e determine 
this by inspecting the number that multiplies the pre-test 
levels in each of the prediction equations. For example, 
at grade 4,' the best statistical prediction of a reading 
post-test level is obtained by multiplying a given pre-test 
score by a factor of .973, then adding this result to the 
constant 1.064. The most important thing to note about the 
prediction equation is that the multiplication factor for 
the pre-test actually specifies the rate of achievement 
growth in post-test scores rela'iive to pre-test achievement. 
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Table 1II-7 



Standardized Achievement Tests Administered In 'Each District and Grade 



District 



'Grade 



Col ton Unified 
Compton Unified 



Fontana Unified 
Garvey Elementary 



L»A» County Special 

Ontario-Montclai r 
Elementary 



1S7^ Pretest 



3 


CTP-23A 




GTBS-Q2 


1 


CTBS-SA 


2 


CTBS-SB 


3 


CTB5-SC 


h 


CTBS-Sl 


5-6 


CTBS-S2 


k 


CTBS-Sl 


5-6 


CTBS~Q2 


2 


CPT-23A 


3 


CPT-23B 


4-5 


CTBS-Q2 


7 


CTBS-Q2 


7-12 


WRAT 


2 


CPT-23B 


3-4 


CTBS-Ql 


5-6 


CTBS-Q2 


7-8 


CTBS-Q3 



1375 Posttest 



CPT~23B 
CTBS-Q2 

CTBS-SB 
CTBS-S3 
CTBS-SC 
CTBS-Sl 
CTBS-S2 

CTBS-Sl 
CTBS-Q2 

CPT-23A 
CPT-23B 
CTBS-Q2 
CTBS-Q2 

WRAT 

CPT-23A 
CTBS-51 
CTBS-S2 
CTBS-S3 
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Mean Grade Equivalen t g ^i^^-^S^2t3£-iiL-5£Bi^:il9_ ^ 





2 


3 


1.0 


1.2 




Grand Total 



1.4* 



Mean Grade EquivaleQ ^ g^j^S^OrgS^in^ Ma± 




8 1.1 \ 0 1,5* 1.3* 2.1* 2:9* 1.6* 



1. Entries rounded "^v^ ^^ciynal place 

* Meets criterion of ^^^e^^j^ng 1.25 months growth 
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For example, at grade 4, a one-year grade equivalent increase 
at pre- test corresponds to a .794 grade equivalent increase 
at post-test. Thus, whenever the multiplication factor 
for the pre- test is lees than 1.0, it indicates that a 
one-year increase at pre- test corresponds to some lesser 
degree of achievement at post-test. Consequently, the 
larger the multiplication factor the greater the achieve- 
ment gain and the greater the impact of the educational 
program. 

The pattern of increasi.ng pre-test multipli.cation 
factors vjith increasing grade levels suggest greater achieve- 
ment growth in the upper grades for standardized reading 
outcomes. A similar, but somewhat less consistent pattern 
is evident for the standardized mathematics outcomes 
appearing in Table III-IO. A glaring exception occurs at 
grade 2, where a one-year grade equi^/alent increase at pre- 
test corresponds to a .159 grade equivalent decrease at 
post- test. We should accept this result with great caution 
since only 26 students provided the basis for this analysis. 
Such a small s-imple si2:e can hardly be deemed representative 
of the entire ALL WIN- USA program. 

That greater pre-tes t/por.t-test growth generally 
occurs at higher grade levels is a finding that is consistent 
with the results of the previously reported AY.L WIN evaluation 
(q.v.. Table III-8) . We speculate, tentatively, tliat the 
poorer achievement growth observed at lower grade levels may 
result from the younger child's more limited ability to work 
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wich complex curricular products. Of course, we possess 
no empirical evidence to substantiate this speculation, 
but V7e offer it merely as an educated guess about the cause 
for the results of the standardized achievement analysis. 

The use of regression analysis to examine pre-test/ 
post-test achievement relations would provide an excellent 
foundation for statistically sophisticated cost-effectiveness 
because constraints in the original sampling plan did not 
permit the kind of representative sampling that cost- 
effective analysis necessitates. Examination of the last 
column in Tables III-9 and III-IO reveal a wide variation in 
the number of students tested in reading (174 to 786 
students) and mathematics (16 to 198) . Such variations 
would, unfortunately, render some of the cost-effectiveness 
ratios (based on predicted outcome residuals) unreliable. 
As a consequence, we utilize the best alternative procedure 
to provide informative cost-effectiveness results for the 
ALL WIN program. This approach involved cos t- effectiveness 
ratios that are computationally based upon cost per gross 
outcome unit utilizing the data from Tabic 111-8. 
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CHAPTER IV 



COST EFFECTIVENESS MALYSIS 
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This chapter is concerned with producing cost- 
effectiveness ratios for the ALL WIN-USA program. These 
ratios will be produced for both reading and mathematics 
and for each of the grade levels for which appropriate 
data are available, . We will consider the fiscal data 
produced in Chapter II and the outcome data of Chapter III- 
-Each win be ex-amlned and madified as necessary to bring 
the two sets of data into conformity. Next, the cost- 
effectiveness ratios will be calculated. Finally, we will 
report on the procedures followed in attempting to attain 
cost-effectiveness data from other projects or programs 
that might , be used for base-line comparison purposes. 

Fiscal Data 

In Chapter II, we have indicated procedures utilized 
for determining the costs of the ALL WIN program. Calcula- 
tions have been made based upon the various data sources -and 
in a succession of tables we have indicated quite clearly 
the manner in which each calculation was performed. Based 
upon this meticulous fiscal analysis, precise cost data 
for the program ha^^s been determined.'^ Ihree categories of 
cost have been isolated: 

1. District expenditures directly related to the ALL V7IN 
program. 

2. Expenditures made directly from the grant by Los Angeles 
County Schools; ^ind 

Under the assumption of the accuracy of the data 
reported by local school districts. 
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3. Regular program costs. 

Examining the first of these, district expenditures 
directly related to the operation of the ALL WIN program, we 
note that the results of our calculations have determined 
this figure to be $143.00 per student in the program. This 
was determined by dividing the expenditures in this category 
from -Table 11-- 15, Coluim 7 by the total number of daily 
students in the program (7,096). We have performed these 
calculations excluding the Los Angeles County Special Schools 
data for the reasons /idicated within Chapter 2. Included 
within this $143 per suudent expenditure^ are the costs of 
teacher ^s salaries, aides, administrative costs, employee 
benefits, regular supplies and expenses, and the maintenance 
and operations costs attributable to the space utilised by 
the program. (As previously noted, this was determined on 
a student pro-rata basis.) 

The second category of fiscal data is expenditures 
made directly through the contract as supervised by Los 
Angeles County Schools. These costs include project admin- 
istr^t'ion by the Los Angeles County Schools, pre-servicc 
and in-service training, evaluation, 5,nstructional supplies 
and equipment (appropriately amortiEed) . The per pupil 
average cost for this category of expenditure is $102.00. 
This figure was determined by divxcling $724,031 (Coltimn 4 
of Table 11-22) by 7,096 ^ olum.n 6 of Table 11-22). The 
total of the above mentioned items 'gives a sum ($245) which 
may be considered an indication of the supplemental Gxpendi- 
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tures .for the program. 

However, students 5.n all cases are in regular 
classrooms and attend the ALL WIN Center for only a portion 
of their total school day. Thus, the total cost of education 
for each student must include the regular program costs of 
providing education in the indicated school districts as 
supplemented by the $245.00 additional cost incurred from 
the two sources mentioned above. Calculations from Table 
11-21 of Chapter 21 yield an average regular program cost 
for the districts involved (excluding Los Angeles County 
Special Schools) of $1,182 per student in Average Daily 
Attendance . 

A closer examination of the actual practices thit 
takes place xrLuliln tht-, school districts reveals a fux^ther 
difficulty with the fiscal data. While each of the school 
districts reported teachers^ salaries associated with 
operating the Aj^L WIN-USA Learning Centers, apparently this 
does not represent- an incremental teacher salary above and 
beyond that which would normally be provided for the number 
of stijdt=»nts vStr-pulated. V/hetber it w^p: the TT^tent of th(^. 
program that there, be an additional teacher, we do not knov;. 
But based upon our observations and follow-up conversation 
with Dr. Harry Holmberg, we have been assured that what 
happens in actuality is that through a process of students 
being assigned to teachers for various hours (e.g. the ALL 
WIN Learning Center is a homeroom for a group of students) 
no - incremental teacher cost is incurred. Thus, while the 

IV-3 

82 



data reported by*the districts and the program, as we under- 
stood it, seemed to imply the necessity of calculating 
teachers' salaries as an additional district expenditure, 
apparently there is the need to modify those fiscal data to 
concur with the reality of the situation. The expenditures 
for teacht^rs ' salaries indicated as spent directly by the 
■districts (Table II-3) is $564,847.00. The expenditure for 
substitute teachers which need not be included since it is, 
by this line of reasoning, already a part of the regular 
program expenses of the district, is $8,092.00 (Table II-5) 
and the associated health and welfare benefits for the 
teaching personnel excluded (Table II-8) is $27,973.00. Thes 
total $600,912.00 in excluded expenditures for all districts 
except Lcs Angeles County Special Schools , or a total of 
$84.68 per student. 

VJhen the $84.68 per student is deleted, this reduces 
the direct district expenditure per student from $143.00 
to $58.00, which wheii added to the $102.00 per pupil average 
cost directly spent by the contract yields a total of $160.00 
per pupil in stipplemental costs. A fxirther comment on the 
subject of the actual practice of the school districts and 
potential areas of conf lict--while some instructional aides 
in the districts may have been paid for by "other*^ grant 
funds and not directly chargeable to ALL WIN, this practice 
is by no means uniform, and, in any case, seems to be 
appropriately listed as a direct district expenditure for 
the program. Likewise, all other costs charged to the direct 
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expenditure category, including proportionate share of 
administrative expenses, related benefits, instructional 
supplies and materials, all appear to be quite appropriate. 
Thus, to summarize, based upon actual practice in the 
districts involved, and not necessarily upon the intent 
of the program, the total of the direct expenditures - 
•incurred either by -the district or through the contract, 
is $160.00 per student. 

However, the inclusion of $600,912.00 from district 
direct expenditures, increases the regular district 
expenditure tabulation of Table 11-21 (Column 4) by a like 
amount. The net effect of this addition is to increase our 
estimate of the average regular program costs for the 
districts involved by $7.00 per student, to $1,189.00. 
Tliis is summarized in Table IV- 1, which follows. 

Ou tcom.0 Data 

The outcome data are presented in complete form in 
Chapter III. The results presented in that chapter are not 
in need of adjustment in order to be compatible with the 
fiscal data as modified. Thus, v/e v/ill simply present a 
summary of the pre-test/pos t- test differences as indicated 
in Chapter III. They are summarized by grade level in 
Table IV-2, following. 
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TABLE IV- 1 

Expenditure Siommary (per ALL WIN Student) : 
Before and After Teacher Salary Modification 





Before 
Modification 


After 
Modification 


Direct District 
Expenditures 


$143.00 


$58.00 


Expenditures Made 
by Grant 


$102.00 


$102.00 


SUB TOTAL 


$245.00. 


$160.00 


Regular Program 

District Expenditures 


$1,182.00 


$1,189.00 


TOTAL 


$1,427.00 


$1,349.00 



TABLE IV- 2 

Pre-Test/Post-Test Differences 
(By Grade Level) 



GRADE 


READING 


MATH 


2 


1.0 


. 0.8 


3 


1 . 2 


• 1.1 


4 


1.7 


1.0 


5 


1.6 


1.5 


6 


1.3 


1.3 


7 




2.1 


8 




2.9 
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Since L.A. County Special Schools fiscal data have 
been excluded, it is necessary to be sure that Special 
Schools test data also are excluded from the cost effect- 
iveness analysis. Fortunately, these schools were not 
included in the test sample used in this study because 
they used a vastly different Reading Test. This omission 
insures a "cleaner" comparison of fiscal and outcome data. 

Cost Effectiveness Ratios 

The fiscal data were collected on a basis aggre- 
gated by district; thus, there is no breakdown as to the 
expenses incurred at each specific grade level within 
district or incurred at a grade level basis across districts 
Given the time of funding of the project and the necessity 
for making great demands upon district personnel durin.q; 
the difficult vacation period, we felt it best not to 
impose the additional request of itemized breakdovm of 
costs by individual class. Thus, we are unable to deter- 
mine different cost figures for different grade levels and 
must use the aggregate data discussed above as indicators 
at all grade levels. Fort-anately > the tost data to be 
utilized in this analysis are only available up to the 8th 
grade and our examination of fiscal patterns in the school 
districts of California reveals that greatest cost increases 
occur past that level. Thus, the use of the same aggre- 
gated cost total for each grade level is not viev/ed as a 
severe disadvantage. 

Using the fiscal cos5ts directly incurred by the. 
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program ($160.00 per student) and the regular program • 
costs ($1,189-00 per student) in comparison with the 
student test gains reported in Table IV-2, we are able to 
determine cost effectiveness ratios for each grade level. 
These cos*t effectiveness ratios are, in essence, the cost 
per month gain in student achievement for every month 
students are in the program (cost per month's gain per 
month in program) . These calculations are shovm in 
Tables IV-3 and IV-4. 

Com parison Data 

Asi previously noted in Chapter I, there are no 
comparison programs available within the school districts 
in which the ALL WIN-USA program was implemented. Thus, 
in order to make any kind of comparisons of cost effect- 
iveness ratios, even crude one.s , it was necessary to 
generate baseline data from other sources . Nothing was 
. available currently from either the California State 

.Department: of Educatiion, Office of Planning and Evaluation, 
or the Office of Evaluation of the U.S,O.E., that would be 
of assistance in this study. The study team conducted 
an extensive computer search of the literatu?:e on compensa- 
tory education using a number of descriptors de;^ igned to 
potentially provide information on the cost ef f ectivenes'j 
of compensatory education programs. This procedure, which 
is described more fully in Appendix 2, yielded 681 refer- 
ences V7hich were tl -n systematically examined by the 
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TABLE IV- 3 



ALL WIN Cost Effectiveness Ratios: Reading 
(by Grade Level) ■ 



C/E Ratios fo" C/E Ratios for 

Grade Supplemental Cc . Total Coets 



$160 _ $1.349 



1.0 



= 160 ■ = 1,349 



160„ ^_ -J^t^ = 1.124 



^60_ ^ -^T^ = 794 



160^ 
"1.6 



= 100 



1.349 
1.6 



8 - 1 



150 
1.3 



= 123 



1.349 

~T73~ 



= 1.038 
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TABLE IV- 4 



ALL WIN Cost Effectiveness Ratios: Mathematics 
(by Grade Level) • 



C/E Ratios for C/E Ratios for 

Grade Supplemental $ Total $ 



200 1,686 



160 1,349 , ooc 

— -j^^ Y 1^^ — l!l — 1,226 



160 1 , 349 T -,,Q 

10 -^"^ — TTO — 1,349 



■if^= 107 899 



-1^= 1,038 



76 642 



,160 1,349 _ 

2.9 ' 279 
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ev£^.luation staff in a lengthy period of . library research. 
A large nximber of the more comprehensive studies in terms 
of the description of costs as well as outcomes of com- 
pensatory education programs were found in Georgia. 
Apparently, state laws there require the inclusion of 
certain fiscal data in the evaluation reports. However, 
the school accotinting practices in Georgia Vc?.ry substan- 
tially from those in California and those reports were 
found to be not appropriate for use for comparison purposes. 

The studies found through the course of this 
extensive investigation to have ^he greatest potential 
relevance for use in comparison in this study, were two 
California Title I reports for fiscal 1972. One, the Annus:^ 
Report f or the Evaluat ion of ESEA Title I Projec ts _of 
California Schools 1971-72 , provided average gain scores 
by grade and average dollars per student spent on th^ 
program statewide. These data proved to be too highly 
aggregated to be of value for comparison purposes. But this 
report did prove to be of value in helping to establish the 
credibility and correctness of .data in another report found 
through the literature search. We had been - informed by the. 
California State Department of Education officials that ther 
were problems V7ith respect to the uniformity of fiscal 
reports provided in 1971-72, and no attempt at systematic 
collection of fiscal data for cost ef f ectiveit.-^.ss purposes 
has occurred in California co.npensatory edi:cation prograius 
since that time. However, in a study performed by G. Kasten 
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Tallmadge of the /jnerican Institute for Research in the 
Behavioral Sciences, relatively ''clean" cost and student 
achievement data are available for 1971-72. 

First, Tallmadge considered compensatory programs 
as being in either of t^-To categories: ''saturated'* (consist- 
ing of 75% or more of the student population eligi? for 
Title I participation), or "unsaturated." For purposes of 
the ALL WIN comparison, it is more appropriate to use his 
data from saturated schools. For his analysis he utilized 
those schools within each category which had reported both 
achievement data and expenditures in their ESEA Title I 
report of 1971-72. Results were reported by grade level for 
both reading and mathematics . The average monthly gain for 
reading in the saturated programs was 1.04 and the similar 
gain for mathematics was 1.29. These mean results across 
grade levels are summarized in Table IV-5. The comparable 
data by grade level are summarized in Tables IV-6 and IV-7. 
In these tables, we note that the greatest gain in student 
achievement relative to total expenditure occurred in 
the 7th and 9th grades with high (for the most part) but 
generally unpredictable costs and cost effectiveness ratios 
in grades 10-12, 

It would be inappropriate, hov;ever, to simply compare 
these co-t effectiveness ratios (cost per month's growth) 
with the ALL WIN ratios of the sam.e type since 1971-72 
school costs were quite different fi >m 1974-75 school costs. 
Ttxus, in order to compare (even in some kind of tentative 
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fashion) the ALL WIN cost ef f ecCiveness ratios with the 
1971-72 comparison ratios, it is necessary that they be 
converted to similar dollar equivalencies. An examination 
of the Annual Keport of Financial Transactions Concerning 
School Districts of California for the years 1971-72, 1972- 
73 and 1973-74 provides information on the average total 
current expenses of education per ADA in California fcr 
each of those years. These *:a are sximmarized in Table 
IV- 8. 

TABLT^. IV- 8 

Total Current Expenses of Education 
per ADA in California'^ 



Elementary Percent Unified Percent 
Districts Increase Districts Increase 



71- 72 - $792.10 $ 896. 47 

72- 73 866.61 9% 969.70 8% 

73- 74 985.48 14% 1,083.41 12% 



Conversations with finance personnel in the admin- 
istration section of the California State Department of 
Education reveal that comparable data are not yet available 



Data from Annual Report of Financial Transactions 
Concerning School Districts of California. 1971-72, 1972- 
73, 1973-74. 
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for the year 1974-75. Thus, it is necessary for us to 
estimate the appropriate increase in TCEE, 

Since v/e do not have data readily av .liable on 
the distribution of districts within che Tallmadge study 
that were elementary as opposed to unified districts, we 
have chosen to take a yearly rate of increase equal to the 
average of the percent increases for the two categories 
designated in Table IV- 8. This would amount to an 8%% 
increase to 1972-73, a 137^, increase over the previous year 
to 1973-74. We have arbitrarily selected, based upon 
historical experience as V7ell as our ovm best judgment, 
a rate of incx'ease of 11% to use in the conversion of the 
data to 1974^^75 equivalencies. The application of these 
cost inflators provides 1974-75 cost effectiveness equivalent 
ratio? for each grade le^^'el and in total for the Tallmadge 
data. rhc£:c are shov.M in Table IV- 9. With the effect of 
compounding of the percentage increases (13% of 8%7o, etc.) 
the total p^ircentage necessary to apply to 1971-72 data, 
in order to obtain coniparable 1974-75 data, is ap|.roxirnately 



36%, 



EKLC 



Applying the cost inflators to the California 
Title I 1971-72 data, new ratios are determined which may 
be considered as adjusted to 1974^75 dollar eqiiivalencies . 
Tliese adjusted ratios arc shovm in Table IV-IQ. 

In Tables IV-11 and IV- 12, v/e have displayed the 
cost effectiveness ratios of the California Title I pro- 
ject (adjusted) .m Table IV-10, and the ALL WIN cost 
effectiveness ratios (cost per month^s growth) from Tables 
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and IV- 7. An examination of Table IV- 11 shows lower 
cost effectiveness ratios for ALL WIN relative to the com- 
parison programs at all grade levels other than grade 2. 
At several grade levels, in particular grades 4 and 5, the 
difference in cost effectiveness ratios seems to be quite 
substantial. The higher cost effectiveness ratios in read- 
ing at grade 2 seems to be compatible with the phenomena 
that we have been hypothesizing throughout the course of 
this report; that is, it may well be that ALL WIN materials 
are less effective with younger children because of the 
complexity of their use. This "explanation'* must be con- 
sidered as merely speculation on the part of the researchers 
into possible causes of the disparity. 

In Table IV-12, we have displayed the cost effect- 
iveness ratios for the ALL WIN program and the comparison 
programs for the area of mathematics v;here the results were 
much more mixed. In no instance V7as the supplemental dollar 
cost effectiveness ratio for ALL WIN schools more advan- 
tageous than for the comparison programs. In part, this may 
be related to the generally lov/er level of expenditure for 
mathematics as opposed to reading in Title I. In part, the 
achievement gains on the State averages of the comparison 
programs are higher than for ALL WIN. VThen we turn to an 
examination of the total dollar cost effectiveness ratios 
for mathematics, the picture is somewhat more favorable. 
In this category, the comparison programs have more favor- 
able cost effectiveness ratios in four instances, while the 
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ALL WIN program has more favorable ratios in three instances 
Again, the familiar pattern noted in the earlier description 
of reading persists. That is, the ALL WIN program does less 
well at the early grade levels (grades 2, 3, and 4) and does 
better at the middle grade levels (grades 5, 6, and 8). The 
pattern is broken at grade 7 where the ALL WIN cost effect- 
iveness ratios are higher than the ratio for the comparison 
programs . 

Summary 

When compared to the average cost effectiveness 
ratios of California Title I reading programs, for 1971-72, 
adjusted to a 1974-75 level of cost effort, the ALL WIN 
program appears to have more favorable cost effectiveness 

0.- 

ratios at grades studied other than Grade 2. The consis- 
tency of the finding of generally lower scores and gener- 
ally higher cost effectiveness ratios at early primary 
grades may be related to the nature of the ALL WIN materials . 
The results at grades 3 through 6 would certainly seem to 
be an encouraging indicator of the potential cost effect- 
iveness ratios of ALL WIN reading materials at these grade 
levels. Especially is this true when one considers that 
the ALL WIN program results V7ere achieved during the first 
year of implementation of the program, ordinarily a time 
of considerable chaos. 

The cost effectiveness ratios for mathematics bear 
out the tendency of ALL WIN students to do less V7ell at the 
early grade, levels and consequently all of the cost effective 
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ness ratios tend to be higher. The performance of the ALL 

matheoiatics program at the middle grades yields cost 
effectiveness ratios slightly more advantageous to the . 
ALL WIN program. It would be worth examining the extent 
to which the settling of the program in subsequent years 
modifies the cost effectiveness ratio differences between 
ALL 'WIN and the comparison program. 

Final Note 

As we have indicated throughout this entire report, 
this cost effectiveness study must be considered as primarily 
an indicator of potential cost effectiveness. We have en- 
gaged in an exercise designed to consider the manner in which 
a cost effectiveness study might take place. To the extent 
to which the cost data initially collected are sound,, 
effectiveness data are correct and the comparison districts 
are similar, the results of this study increase in signifi- 
cance. The methodology employed in this study has been 
sound. Hopefully, greater attention to refining procedures 
for cost data (as discussed in Chapter II) should increase 
the trust that can be .placed in the findings of subsequent 
studies . " 
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EDUCATIONAL EVALUATION ASSOCIATES 
9230 Jellico, Northridge, California 91324, (2!3) 



Dear 



As you undoubtedly Icnow, EEA has been conducting an 
evaluation of the ALL-WIN program for the Los Angeles County 
Schools. We have been aslced to expand the study to include' 
cost effectiveness considerations. Certain data is needed 
for' the cost effectiveness study being sponsored hy OEO and 
the Los Angeles County Schools. Would you be kind enough to 
provide the information requested on the enclosed questionnaire 
related to your District's overall costs and to ALL-V/TN costs 
and operation. You may need the assistance of your district's 
business division in providing some of this information. All 
data required should be for the 1974-75 fiscal year. 

If there are any questions about what is needed, please 
feel free to contact me at (213) 825-4800 or (213) 993-8070. 
I will be pleased to provide v;hate\^er clarification is necessary 
or to provide other assistance. 

Please make every effort to return the questionnaire by 
July 28. Time is of the essence and ve do appreciate vour 
assistance . 



Sincerely, 



MCA/k 



Marvin C. Alkin 
Director 



Enclosure 
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Questionnaire on All-Win Financial Data 



1. Please provide information related to those District personnel 

(irrespective of funding sources) serving directly in the All-Win 
program* Please list all. 



Category 



Name oi 
FersonCs) 



School 
Location 



A. 



Central Office Admin • 
Responsible for Program 



School Principal (s) 
Serving in Program 



Classroom Teachers 
Serving in Pro&ram 



Teacher Aides 
Serving in Program 



-Present best estimate possible 



Jgalary^ 



Per Cent 
of Time ^ 
in Prograq 
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What was the district average expenditure per employee for Health and 
Welfare benefits. (Health Insurance, Tax Shelter Annuities, Life In- 
surance, etc.)? 



Certificated Employees 



Classified Employees 



According to district policy do all classified employees receive a 
full Health and Welfare allocation? 



Yes 



No 



If not, what part of the Health and Welfare allocation did the Teacher 
Aides (ID above) receive? 



How much did the District spend directly in the All-Win program for 
supplies and materials? 



How much was spent in the Operations (custodial, grounds, utilities) 
portion of the District budget as determined by the program budget? 



How much was spent in the Maintenance portion of the District budget 
as determined by the program budget? 



Is the District securing classroom facilities under a lease or 
lease/purchase plan specifically for the All-Win program? 



Name 



Amow.c or Percentage 



1. 
2. 
3. 
4. 



Yes 



No 



If ye^, what is the annual payment (s)? 



Per Classroom 



Total for All-Win Facilities 
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7. What was the District's total expenditures for all budget categories 
(1000, 2000, 3000, 4000, 5000 and 6000 excluding 7000)? 



8. How many full-tice equivalent teachers served in the District during 
the 1974-'75 school year? Include all regular, special education, 
special project, and Federally funded teachers in count. 



9- Ii?hat is your daily rate for substitue teachers? 
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DATA NEEDED FOR COST DETERMINATION 



Direct Costs 
Item 



.Determination 



Central Administration 
- Person responsible 
at District level. 



Site Administration 
-Person responsible 
at 'School level 



Teacher serving in 
program 



Aide(s) serving in 
program 



Health and Welfare 
benefits of all parti- 
cipants 



Custodial and Opera- 
tions 



Maintenance 



Supplies and 
Materials 



Retirement 



Percent of actual salary 
as estimated by respondent 



Percent of actual salary 
as estimated by respondent 



Actual salary of partici- 
pant 



Actual salary of partici- 
pant (s) 



Percent of actual fringe 
benefit 



Actual expenditures for 
operations prorated 
according to number of 
work stations 



Actual expenditures for 
maintenance prorated 
according to number of 
work stations • 



Actual expenditures for 
All Win 



Use percent of salaries 
determined in A-D above 



Data Collection 



Questionnaire 



Questionnaire 



Questionnaire 



Questionnaire 



Questionnaire 



Questionnaire 



Questionnaire 



Use State re- 
quired percent 
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Indirect Costs 



Items 


Determination 


Data Collection 


All expenditures for 

the following categories: 


Take total expenditures 
for 1975-75 and subtract 
\r^' following: 


Questionnaire 



1000 Certified 
2000 ClassifieU t.. 
3000 Fringe benefits 
4000 Supplies, Materials 
5000 Contract Services 
6000 Capital outlay 



.0 Debts, transfers 



All costs determined in 
lA-I above 



All lease payments for 
All Win facilities 



Prorate according to 
work stations 
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LITERATURE SEARCH PROCEDURES AND 
ANNOTATED BIBLIOGRAPHY 

The SDC On-line Bibliographic Search Service has 
been developed and operated by System Development Corpora- 
tion. The educational data base for the literature search 
is ERIC ducat ional Resources Information Center) , developed 
and liuL d by the U.S. Office of Education. Currently, 
there aj.. eighteen ERIC clearinghouses located throughout 
the United States that now report to the National Institute 
of Education. They collect, screen and abstract the reports 
and periodical literature in education and education-related 
fields. SDC's ERIC data base covers all citations published 
monthly in Research in Education and Current Index to Journals 
in Education, and it is retrospective to 1966. 

In order to conduct the most efficient and relevant 
search possible, the study team established certain search 
parameters. First, all citations referenced to the 
descriptors "compensatory education," "Title I," "ESEA Title 
I," "cost effectiveness," "student expenditures," and 
"program costs" were located. This produced an initial 
reference file of over 1,400 citations. The second phase 
of the search involved further refinement of this reference 
file, and was conducted by first specifying all descriptors 
associated with higher education. Next, any of the 1,400 
citations referenced to higher education were deleted from 
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the file. This procedure ensured that the final citations 
would not only cover compensatory education, but would be 
restricted to compensatory education at grades K-12. The 
final citation file contained 681 references. Those refer- 
ences which appeared to be most relevant from the reading of 
the abstract were examined with relevant information 
•suramarizred- on the foil owing form. These references have 
beer ^-Gsented to Dr. Har-^^ Ilolmberg for his further 
-iAcA.i; - liuCion. 
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ALL WIN COST EFFECTIVENESS STUDY 
Data Form 



Complementary Ed. program Yes 



What grade level was the 
program ? 



What subject field? 



What kind of outcome test 
was used? 



Gains showr 

(descr-ibe rrJCy) 



Kow much? 



What is iruclluder in costs? 

or how Ji^re they determined?' 



No 



Costs deseiLrx3e;d Yes 



No 



Source : 
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